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Planning"; and (5) "Using Results To Improve Teaching and Learning." (SLD) 
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Student Learning 
Assessment 

Options and Resources 
The Handbook at a Glance 

This handbook from the 

Middle States Commission on Higher Education 
is a bridge between the Commission’s standards for 
accreditation and the practical challenges of 
planning, assessment, and continuous improvement. 



Contents 

1 . Motivating and Involving Campus Communities 

2. Learning Goals 

3. Evaluating Student Learning 

4. The Student Learning Assessment Plan in the 
Context of Institutional Planning 

5. Using Results to Improve Teaching and Learning 



The handbook and its companion website, 

www . msache . org/mainstudents . html . offer extensive resources, 
including exercises, hundreds of examples from assessment practices, 
and guidelines that can be used on campus for self-reflection, 
brainstorming, and workshop exercises. 

Middle States Commission on Higher Education 
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Student Learning Assessment: 
Options and Resources 



User-friendly Resources! 

The explanations and examples in the handbook and website are intended to help 
each institution to develop goals and to select the best methods. They do not increase 
or modify accreditation standards. 

The handbook addresses the needs of everyone interested in, or practicing, higher 
education assessment. Each chapter focuses on a separate aspect of the assessment 
process, so the chapters can be used individually and at any stage of the process as 
a guide and stimulus. 

Useful appendices include: Assessment Standards in Characteristics of Excellence in 
Higher Education ; An Assessment Practices Quiz; Key to "Assessment Practices Quiz"; 

A Department/Program Student Outcomes Survey; Learning Goals and Assessment 
Techniques; From Effect to Cause; A Brainstorming Exercise; and Student Learning 
Styles: Frequently Asked Questions. 

The website is organized by the chapter topics in the book. It provides hundreds of 
resources, including on-line assessment activities and assessment "clearinghouse" sites. 



Chapter 1 : Motivating and Involving Campus Communities 

This chapter offers encouragement, guidance, and resources to those who are 
leading assessment initiatives on their campuses. It offers strategies for chief academic 
officers and leaders of assessment to encourage involvement and "ownership" by 
faculty, staff, and students. 

Institutional culture plays an important role in the success and value of assessment 
efforts. The culture is created through an interaction of both formal and informal 
influences, such as the form of control of the institution (public or private), any 
faith-based affiliation it may have, its governance practices, and the prevailing social 
and political context. The handbook suggests questions that institutions might wish 
to ask as they identify aspects of their culture that may enhance or frustrate 
assessment efforts. 
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Chapter 2: Learning Goals 



This chapter provides many resources for articulating learning goals clearly, and it 
describes the relationships among learning goals at the course, program, and 
institutional levels. It focuses on using clear, straightforward, and meaningful goals. 

It recommends that institutions base their assessment programs on a few important 
goals, rather than on complicated lists of secondary goals. Resources include several 
exercises to help groups of faculty and others to develop shared learning goals, such 
as the Angelo & Cross Teaching Goals Inventory and leading questions for developing 
learning goals. 



Its sections include discussion and examples of: 

benefits that clearly articulated learning goals will have for students, faculty and staff, 
the institution, and the public 

relationships among learning goals at all levels (institutional, program, department, 
and course levels) 

^ first steps toward developing learning goals (situation audit; starting with what already 
exists; examining existing practices for success) 

^ defining learning goals before selecting assessment methods 

ensuring the quality and relevance of learning and goal statements (key learning 
outcomes; wide agreed-upon concepts; communication of learning goals; 
meaningful learning goal statements that lead to improvement; sufficiently explicity 
learning goals; interconnectedness of student learning goals) 

A resources for creating student learning goal statements (existing syllabi and course 
descriptions; leading questions; analysis of student work; inventory of teaching goals) 



One example of the relationship among learning goals at the institution and program 
levels: 



Institutional 

Goals 


Programs and Program-level Goals 


Leadership 


General 

Education 


Residential Life 


Business 


History 


Function 
effectively as a 
team member 


Apply conflict 
resolution skills 


Develop 
leadership skill in 
self & others 


Analyze historical 
perspectives on 
leadership 
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For choosing learning goals, the handbook recommends that institutions: 

Start with success 

Determine which learning goals are already being assessed and what data 
may be available to assess other goals at institutional, program, and course 
levels 

Define learning goals before choosing assessment methods by operationally 
defining each goal and tailoring data collection to defined goals 

Ensure relevance of goals by identifying key learning outcomes, using widely 
agreed upon concepts, communicating goals, selecting important and 
meaningful goals, being explicit, and integrating goals in different areas and 
levels. 



Chapter 3: Evaluating Student Learning 

This chapter provides an overview of the methodological considerations that are 
relevant to assessing student learning. It suggests several easy-to-implement practices 
and offers guidance in choosing appropriate types of assessment, depending on the 
learning goals being developed and the institutional resources that are available. By 
explaining methodological issues relevant to evaluating student learning (including 
how to assure validity and reliability), it helps institutions to select among standardized 
or customized tests, quantitative and qualitative evidence, and direct and indirect 
measures of student learning. 

This chapter includes the following sections: 

Basic Issues in Assessment 

-y- formative and summative assessment 
benchmarking 

Methodological Considerations 

direct and indirect methods for assessing student learning at the course, 
program, and institutional levels; limitations and considerations related to 
indirect methods of evaluation 

quantitative and qualitative evidence 

Mother methodological considerations: 

■ reliability and validity; 

■ pretests, post-tests and longitudinal designs 

■ role of grades in assessment 

■ self report measures 

■ statistical vs. practical significance 
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Key Questions when Choosing Assessment Instruments 

the evidence provided by the instrument linked to important learning 
outcomes? 

4 - Is a standardized instrument appropriate for the learning goals of the 
institution? Is the evaluation method appropriately comprehensive? 

-v-Are important learning outcomes evaluated by multiple means? 

4 - Do questions elicit information that will be useful for making improvements? 
Are they clear and interpreted consistently? Does everyone interpret the 
responses the same way? 

4 - Do the results make sense? 

4 - Are the results corroborated by other evidence? 

-v-Are efforts to use "perfect" research tools balanced with timeliness and 
practicality? 

Is evidence gathered over time and across situation? How much should be 
assessed? 

4 - Are faculty and staff members who are knowledgeable about measurement 
serving as resources for developing assessment instruments? 

Easy-to-lmplement Tools and Techniques 

-v- rubrics or rating scales (e.g., simple checklists; holistic rating scales) 

-v- self-reflection 

-V- rating/comments from internship or research supervision 
A placement rates; test blueprints 

Other Assessment Tools 

multi-dimensional or comprehensive tests 
^ ad hoc surveys and pre-graduation surveys 
-<► focus groups 
portfolios 

retention/graduation rates 

Direct and Indirect Methods for Assessing Student Learning 

The concepts of direct and indirect methods of evaluating student learning are 
often confused with each other and with quantitative (numerically-based) and 
qualitative (narrative-based) forms of information. Direct methods of evaluating 
student learning are those that provide evidence that a student has command 
of a specific subject, content area, or skill, or that the student's work 
demonstrates a specific quality such as creativity, analysis, or synthesis. In 
contrast, indirect methods of evaluating student learning involve data that are 
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related to the act of learning, such as factors that predict or mediate learning 
or perceptions about learning, but that do not relate directly to the learning 
itself. Both of these forms of evaluation are valuable, because one determines 
whether a student has learned, and the other explains why a student has 
learned. 



The handbook gives extensive examples of direct and indirect learning at each level. 
For example: 



Direct Measures (Excerpts) 


Indirect Measures (Excerpts) 


Course Level: 

■ Examinations and quizzes 

■ Observations of field work, internship performance, 
service learning, or clinical experiences 


■ Course evaluations 

■ Percent of class time spent in active learning 

■ Number of student hours spent at intellectual or 
cultural activities related to the course 


Program Level: 

■ Capstone projects, senior theses, exhibits, or 
performance 

■ Pass rates or scores on licensure, certification, or 
subject area tests 

■ Employer and internship supervisor ratings of 
students' performance 


■ Registration or course enrollment information 

■ Employer or alumni surveys 

■ Student perception surveys 


Institutional Level: 

■ Performance on tests of writing, critical thinking, or 
general knowledge 

■ Rating-scale scores for class assignments in 
General Education 


■ Transcript studies that examine patterns and trends 
of course selection and grading 

■ The institution's annual reports, including institutional 
benchmarks and graduation rates 



The chapter also summarizes evaluating student learning: 

4 - Use indirect measures to explain or support findings from direct measures 

^Choose the most relevant level for evaluation of the learning goals: 
institutional, program, or course 

Select quantitative or qualitative measures based on the type of student 
learning goals 

4 - Relate goals to learning 

Choose appropriate research design 

^Use formative assessment "mid-course" to improve teaching and learning 
^Use common sense: Is the result logical? 
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Chapter 4: The Student Learning Assessment Plan 
in the Context of Institutional Assessment 



This chapter of the handbook is the starting point for institutions that are beginning to 
develop plans for the assessment of student learning at the institutional and program 
levels, or those that are updating already existing plans. It places student learning 
assessment in the institutional context of strategic planning and broad institutional 
assessment, and it provides a framework for developing assessment plans. 

Its sections include: 

-^Assessment of student learning within an institutional context 
Characteristics of effective assessment plans 

Components of institutional-level plans and program-level plans for the 
assessment of student learning 

The handbook illustrates how each department can address the same issues as the 
whole institution from the department's own perspective, within the context of 
institutional goals. Conversely, it discusses how the institution's overall plan may 
amalgamate those of its several departments when appropriate. 



Chapter 5: Using Results to Improve Teaching and Learning 

This chapter provides examples of how intentional changes at the course, program, 
and institutional levels can produce results that improve learning. A "Message to 
Faculty" suggests clear, practical changes that faculty members can effect to 
improve learning. This chapter is also intended to help institutions to gauge whether 
their environments are conducive to making use of assessment results. 

The excerpts in the table on the facing page are from materials in the handbook that 
illustrate how assessment results can be used at the institutional, program, and course 
levels. 

A Message to Faculty 

The handbook contains a message to faculty on the interconnectedness of 
assessment, teaching, and learning. It emphasizes that improving student learning . 
requires more than creating plans and choosing measures. It begins with the 
faculty-student relationship and the essential characteristics of an environment 
conducive to learning. The handbook suggests ways that faculty members can use 
basic tenets about how students learn in their everyday interactions with students. 




7 



9 



Learning 

Goal: 

Students will... 


Direct & 
Indirect 
Measures 


Outcome 


Possible Reason 
or Hypothesis 


Action Taken 


Institutional 

Level 

Exhibit competitive 
career potential 


Alumni survey 

Data from 

benchmark 

institutions 


Graduates’ salaries 
five years post 
graduation are lower 
than those of students 
with comparable 
positions who 
attended other 
institutions. 


Students are not 
trained in salary 
negotiation. 

Students are under- 
prepared relative to 
other Institutions. 


Offer opportunities for mock 
interview and mock raise 
requests. 

Survey employers to 
determine cause. 

Change curricula as a result 
of employer survey. 


Program Level 

Communicate 
competently in the 
major 


Scores on faculty- 
developed rating 
scales for final 
oral exam and 
final report in 
capstone course 


Student performance 
in capstone courses is 
poor, as measured by 
rating scales for oral 
presentations and 
written reports. 


Students are not 
receiving enough 
experience making 
oral presentations in 
prerequisite major 
courses. 


Revamp departmental 
curriculum to require oral and 
written reports in every 
course. 

Revamp syllabus of at least 
one required course 
to include multiple 
communication experiences. 


Course Level 

Demonstrate 

discipline-specific 

information 

literacy 


Rubric-scored 
term paper 

Student survey or 

course 

evaluations 


Research skills 
are weak, as 
demonstrated by 
poor performance on 
research term paper. 


Expectations for term 
paper were not clearly 
spelled out. 

Appropriate search 
engines or sources 
were not available in 
the library. 

Little practice with 
these skills on smaller 
assignments. 


Create a “blueprint" for the 
paper, clearly spelling out 
expectations. 

Require an early non-graded 
first draft and provide 
feedback. 

Examine library funding for 
possible reallocation. 

Revamp required course 
to include several practice 
assignments drawing on 
scholarly research skiils in 
the discipline. 
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